Suggested Books

One of the major goals of dlib is to have documentation that enables someone to easily make use of its various components. Ideally, you would read a short description of something, understand it immediately, and begin using it in your application without any difficulty. Obviously, this depends partly on the background of the user. For example, if you have never written C++ software before then it probably isn't going to be this easy.

This page is meant to complement the main library documentation by providing references to books, along with my commentary, which explain most of the background material needed to understand the various parts of the library. In most cases these are the books I learned from during the process of creating dlib. As always, if you disagree with anything or think I have left out an important text then shoot me an email.

## General Programming

### C++

- Programming: Principles and Practice Using C++ by Bjarne Stroustrup
  This is the sort of book you would use in a freshman introduction-to-programming class. So if you are just beginning to study programming and are interested in C++ then I think it is probably safe to say this is one of the best books you could read.
- Accelerated C++: Practical Programming by Example by Andrew Koenig and Barbara E. Moo
  If you are new to C++ but already know how to program then this is a great book. It's also about one fourth the size of the Stroustrup book.
- Effective C++: 55 Specific Ways to Improve Your Programs and Designs (3rd Edition) by Scott Meyers
  This is a great intermediate level C++ book. Most people have heard the jokes about how easy it is to shoot yourself in the foot with C++. This book explains many things you need to know about the language to avoid doing so on a regular basis. So if you are writing C++ software then this is a must-read. I would even claim that you are a danger to the C++ software you touch unless you know what is in this book. I'm not kidding. Finally, the book isn't just about the quirks of C++. It also discusses many general software engineering ideas which have wide applicability. So in this respect it is a great book for any software developer to read.
- More Effective C++: 35 New Ways to Improve Your Programs and Designs by Scott Meyers
  Consider this an expansion to Effective C++. If you are going to read the above book then you would almost certainly benefit from reading this one as well.
- The C++ Standard Library: A Tutorial and Reference by Nicolai M. Josuttis
  If you are going to buy a reference book on the C++ standard library then this is the one to get. That is, if you are looking for an in depth reference this is an excellent resource.
- Online C++ Standard Reference
  For a quick reference to just about any part of C++ or its standard library this is an excellent online resource.

### Multithreading

- Programming with POSIX Threads by David R. Butenhof
  When I was an undergrad, this book was my main resource for learning about multithreading. It was enjoyable to read, as are all the books on this list, and covered everything in great depth without becoming overbearing. Also, despite what the title may suggest, this book is useful for understanding multithreading broadly, not just multithreading on POSIX systems.

### Network Programming

- Unix Network Programming, Volume 1: The Sockets Networking API (3rd Edition) by W. Richard Stevens
  A lot of people call this book the network programming Bible and this praise is well deserved. If you want a deep understanding of how computer networks function, including the Internet, then this is the book to read. As with the Butenhof book above, this is an excellent choice even for people who do not intend to write software for Unix systems.

### WIN32 Programming

It has been a long time since I needed to refer to these two books. However, they contained information I couldn't find elsewhere no matter how hard I looked. So I recommend them in case you need to create or understand some low level win32 code.

- Win32 Programming by Brent E. Rector and Joseph M. Newcomer
- Programming Windows by Charles Petzold
- MSDN Library
  This is Microsoft's online reference documentation. It is very large and sometimes confusing. But at the end of the day you should be able to find the documentation for just about every function in the entire Windows API.

## Computer Science: Algorithms and Data Structures

- Introduction to Algorithms by Cormen, Leiserson, Rivest and Stein
  You should get this book if you are looking for a good discussion of the classic computer science algorithms and data structures (e.g. most of the components on the containers page).
- Algorithms in C++, Parts 1-4: Fundamentals, Data Structure, Sorting, Searching (3rd Edition) by Robert Sedgewick
  This is another good algorithms book. If you are going to get only one book on this subject then get the one above. However, when I was learning about these topics I used both these books and on many occasions I found it helpful to read the description of an algorithm or data structure in both. Where one description was a little vague or confusing the other generally filled in the gaps.

## Lossless Data Compression

- Text Compression by Bell, Cleary, and Witten
  When I was studying data compression this was my most useful resource. If you are looking to understand how lossless data compression algorithms work then this is the book you want. It is completely self-contained and an absolute joy to read. Note that contrary to one of the reviews on amazon.com, the book Managing Gigabytes is not the second edition of this book; if this topic interests you then be sure you get the 318 page book published in 1990.

## General Math

- Linear Algebra Done Right by Sheldon Jay Axler
  If a matrix seems like an arbitrary grid of numbers or you find that you are confused by vectors, matrices, and the various things that get done with them then this book will change your whole view of this subject. It doesn't teach you any algorithms. Instead, it will give you a general framework in which to think about all this stuff. Once you have that down everything else will start to make a lot more sense. If all goes well you will even start to agree with the following: linear algebra is beautiful. :)
- Numerical Linear Algebra by Trefethen and Bau
  While Linear Algebra Done Right is fairly abstract, this book by Trefethen and Bau will explain some of the actual algorithms that are often used. This is a great second book if you find that you want to know more about the SVD, LU decomposition, or various other algorithms involving linear algebra.
- Calculus: Single and Multivariable by Hughes-Hallett, Gleason, and McCallum
  Some of the books below will require and understanding of basic calculus. So I'm recommending this book. It was the book I used as an undergrad and I remember it being alright. That isn't exactly a glowing review so if you are really considering buying a calculus book you may want to check out other reviews before picking this one.
- Introduction to Real Analysis (third edition) by Bartle and Sherbert
  At some level real analysis is like a really rigorous repeat of calculus. So if you already have an undergraduate education in calculus and you are reading things that seem reminiscent of calculus but involve stuff you haven't seen before (e.g. sup, inf, "sets of numbers", sequences of points) then you may be in need of a real analysis book. This one is quite good and should be accessible to someone with the usual undergraduate computer science math background.

## Optimization

The subject of linear algebra is fundamental to optimization. So you must be familiar with the contents of a book like Linear Algebra Done Right if you are going to study this area. You will also need to know how to find the derivative of a function and understand what a derivative is all about. So you will need to know a little bit of calculus. Finally, once in a while you will need to know a little bit about real analysis. Ultimately, what you need all depends on how deep you want to go.

- Practical Methods of Optimization (second edition) by R. Fletcher 1987
  I love this book. When I got it I literally spent my weekends sitting around reading it for hours. It is a fascinating and well written introduction to the subject of optimization. This has been my most valuable resource for learning the fundamentals of optimization and I cannot recommend it highly enough.
- Numerical Optimization by Jorge Nocedal and Stephen Wright 2006
  This is a more recent text on optimization that is also very good. It covers many algorithms not covered by the above book.
- Introduction to Derivative-Free Optimization by Conn, Scheinberg, and Vicente
  If you want to understand algorithms like BOBYQA then this is a good recent book on the subject. Note that a book like Practical Methods of Optimization is almost certainly a prerequisite for reading this book. As an aside, BOBYQA is not discussed in this book but its predecessor, NEWUOA is.

## Machine Learning

- Artificial Intelligence: A Modern Approach (3rd Edition) by Stuart Russell and Peter Norvig
  This book is about the much broader field of AI but it contains an excellent introduction to machine learning and it also covers other useful topics like bayesian networks. Moreover, it is very well written and self-contained. So you don't need any particular background to be able to learn from it apart from a typical undergraduate background in computer science.
- Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond by Bernhard Schlkopf and Alexander J. Smola
  Most of the machine learning tools in dlib are implementations of various kernel methods. So if you want a book that covers this topic in great depth as well as breadth then this is probably the book for you. The most important prerequisite for this book is linear algebra. Virtually everything in this book depends on linear algebra in a fundamental way.

  The second important subject is optimization. Whenever you see the text mention the KKT conditions, duality, "primal variables", or quadratic programming it is talking about ideas from optimization. A good book which will explain all this to you is Practical Methods of Optimization. Note that this book calls the KKT conditions just the "KT" conditions. It is talking about the same thing. Also, duality is something that comes up a lot in optimization but in the context of machine learning usually people are talking about a particular form known as the Wolfe Dual.

  It would also be good (but maybe not critical depending on which parts you want to read) to be familiar with real analysis.
- Kernel Methods for Pattern Analysis by John Shawe-Taylor and Nello Cristianini
  This is another good book about kernel methods. If you have to choose between this book and Learning with Kernels I would go with Learning with Kernels. However, it is good to have both since reading different presentations of difficult subjects usually makes learning them easier.
- Structured Prediction and Learning in Computer Vision by Sebastian Nowozin and Christoph H. Lampert 2011
  If you are looking for a book discussing the background material necessary for understanding things like the Structural SVM tools in dlib then this is a good book. It is also available online in PDF form.

## Image Processing

- Digital Image Processing by Rafael C. Gonzalez and Richard E. Woods
  This is a terrific introduction to digital image processing. By and large this book doesn't require any special prerequisites. Sometimes calculus shows up, but not too much.